Skip to content

Don't reload proc macros on fork #141276

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 1 commit into from

Conversation

Berrysoft
Copy link
Contributor

I come across this problem recently, and see the mailing list of cygwin: https://inbox.sourceware.org/cygwin/TYCPR01MB10926F334CDC9344A89527583F89CA@TYCPR01MB10926.jpnprd01.prod.outlook.com/

TL;DR: rustc fails to execute the linker when proc macro DLLs are too many.

As we don't need to reload these them when executing the linker, I think it's OK to set FORK_NO_RELOAD for the loaded proc macro DLLs. It's a little hacky though. dlfork is a cygwin extension without documents, and it cannot set whether to reload a specific DLL on fork. If available, I prefer such an option for dlopen.

Not sure what does the rustc maintainers think about this workaround. I'll also ask for advice from the cygwin maintainers.

@rustbot
Copy link
Collaborator

rustbot commented May 20, 2025

r? @oli-obk

rustbot has assigned @oli-obk.
They will have a look at your PR within the next two weeks and either review your PR or reassign to another reviewer.

Use r? to explicitly pick a reviewer

@rustbot rustbot added S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue. labels May 20, 2025
@jeremyd2019
Copy link
Contributor

This is a fairly dangerous workaround unless you really know everything going on in your process. I think it would be safe if:

  1. no dynamically-loaded DLL calls fork, and
  2. no call is made into a dynamically-loaded DLL in the child process until it exits or calls exec.

I think we really need to debug this in Cygwin and figure out what's going on, because it seems at this point like a "can't happen" case is happening.

@Berrysoft Berrysoft marked this pull request as draft May 20, 2025 18:08
@Berrysoft
Copy link
Contributor Author

OK. I've changed the PR to draft. Hope it won't take long time to figure out the cause.

The child processes of rustc seems no reason to call into the proc macros. However, the proc macro itself might call fork to execute other executables. In most cases, there's no reason for a proc macro to create another process, but rustc allows to do so.

@bjorn3
Copy link
Member

bjorn3 commented May 20, 2025

Rustc can also load external codegen backends as dlls, which can and do spawn new processes (for example for linking), which internally does a fork. Rustc may decide to codegen the entirety of Command::spawn inside the codegen backend dll.

@jeremyd2019
Copy link
Contributor

Yeah, that'd be bad. It seems like in most cases it'd probably use posix_spawn, and that would be ok (the fork and exec would be contained inside the cygwin dll). Still, not a safe thing to do in general.

@Berrysoft
Copy link
Contributor Author

Cannot reproduce this error in the latest master branch. Close now.

@Berrysoft Berrysoft closed this May 21, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
S-waiting-on-review Status: Awaiting review from the assignee but also interested parties. T-compiler Relevant to the compiler team, which will review and decide on the PR/issue.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants